Increasing Weak Classifiers Diversity by Omics Networks

The common problems in machine learning from omics data are the scarcity of samples, the high number of features and their complex interaction structure. The models built solely from measured data often suffer from overfitting. One of possible methods dealing with overfitting is to use prior knowledge for regularization. This work analyzes contribution of feature interaction networks in regularization of ensemble classifiers representing another approach to overfitting reduction. We study how utilization of feature interaction networks influences diversity of weak classifiers and thus accuracy of the resulting ensemble model. The network and its random walks are used to control the feature randomization during construction of weak classifiers, which makes them more diverse than in the well-known random forest. We experiment with different types of weak classifiers (trees, logistic regression, näıve Bayes) and different random walk lengths and demonstrate that diversity of weak classifiers grows with increasing network locality of weak classifiers.